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Abstract 


Consider  a  communication  channel  without  feedback,  with  transmitted 

signal  A(X) ,  and  with  additive  Gaussian  noise  N.  The  information  capacity 

2 

of  this  channel  is  obtained  subject  to  the  constraints  E||A(X)||  s  P,  where 

w 

11*11^  can  be  regarded  as  the  RKHS  norm  of  the  stochastic  process  W.  The 
class  of  admissible  processes  W  includes  all  Gaussian  processes  having  in¬ 
duced  measure  equivalent  to  the  measure  induced  by  N. 
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Introduction 

This  paper  considers  the  information  capacity  of  the  Gaussian  channel  without 
feedback.  The  channel  input  is  a  sample  function  from  a  stochastic  process  X;  it 
is  encoded  into  the  signal  sample  function  by  a  coding  operation  A;  the  channel 
adds  a  sample  function  from  a  Gaussian  noise  process  N  (independent  of  X);  the  chan¬ 
nel  output  is  then  a  sample  function  from  the  process  Y  =  A(X)  +  N.  The  quantity 
of  interest  is  the  information  capacity 

C  =  sup  I (X, A(X)  +  N] 

a 

where  I [U,V]  is  the  mutual  information  of  the  processes  U  and  V,  and  Q  is  a  set 
of  (A, X)  defined  by  appropriate  constraints. 

Let  ll*liN  denote  the  reproducing  kernel  Hilbert  space  (RKHS)  of  N;  under  the 
constraint  e||A(X)||n  <  P,  a  complete  solution  to  the  information  capacity  problem 
is  given  in  [1].  However,  there  is  also  considerable  interest  in  the  capacity  prob¬ 
lem  using  constraints  of  a  different  type.  The  following  constraint  has  been  ex¬ 
amined  in  special  cases  in  various  publications  [2],  [3],  [4],  [6]:  £  is  the 
set  of  all  (A, X)  such  that  E| |A(X) | s  P,  where  W  is  a  second  Gaussian  process. 

There  are  various  motivations  for  this  definition  of  <£•  For  example,  if  W  is  the 

t  2 

Wiener  process  on  [0,T],  as  in  [2],  and  [A(X)]t  *  /  V  ds,  then  | |A(X) | L 
T  0  s 

■  /  vfdt.  This  is  the  usual  power  constraint.  A  more  general  motivation  is  that 
0  z 

in  many  applications  one  may  not  know  the  precise  covariance  of  N.  It  is  then  of 
interest  to  calculate  upper  and  lower  bounds  for  the  capacity,  over  all  Gaussian 
processes  N  whose  induced  measures  are  mutually  absolutely  continuous  to  the  mea¬ 
sure  induced  by  some  "most  likely"  (or  reference)  Gaussian  process  W. 
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A  complete  solution  is  obtained  here  for  the  capacity  problem  using  the 
above  definition  of  In  a  subsequent  publication,  the  results  obtained  here 
will  be  applied  in  an  analysis  of  the  feedback  channel  of  [2]. 

Mathematical  Structure 

The  channel  model  is  defined  as  in  [1].  The  channel  noise  N  is  represented 
by  a  measure  uN  on  the  Borel  o-field  of  a  real  separable  Hilbert  space  H y  The 
message  X  is  modeled  by  a  measure  yx  on  a  real  separable  Hilbert  space  H^.  The 
inner  product  on  H^  is  denoted  by  <-,*>^.  A:  H^-*-  H^  is  a  Borel-measurable  coding 
function.  yw  is  a  strong  second  order  measure  on  H2  (JH  |  |x|  l^dy^x)  <  ») .  R^  and  R)tf 
denote  the  covariance  operators  of  y^  and  y^.  The  following  assumptions  are  made. 

1/2  1/2 

(1)  Rj^  =  Rw  (I+S)RW  ,  where  S  is  a  compact  operator  in  H2,  and  S  does 
not  have  -1  as  an  eigenvalue.  Without  loss  of  generality,  it  is  assumed 
that  range (Rn)  =  H2- 

(2)  The  admissible  set  £  is  the  set  of  all  (A,yx)  such  that 

JH  I Irw1/2a(x) I \\  dyxCx)  s  P. 

The  definition  of  Q.  given  in  (2)  implies  that  yx  •  A-1  is  strong  second  order. 

Since  we  can  (and  do)  assume  that  y^,  y^,  and  yx  o  A  1  each  have  zero  mean,  the  con- 

-1/2  -1/2 

straint  in  (2)  is  equivalent  to  trace  R^  ^AfX)^  s  where  is  t*ie  co" 

- 1/2 

variance  operator  of  A(X).  Note  that  Rw  exists  because  of  assumption  fl) •  If 
range (R^  +  H2  in  a  given  problem,  one  can  replace  H2  WLOG  by  range (R^) .  Alterna¬ 
tively,  one  could  use  the  original  H2  and  apply  the  constraint 

JH  I  6nJ  <A(x).2n>2  dyx(x)  s  P,  where  R^  -  \  «nZn  •  Zn,  with  each  6n  >  0  and 
2  n  n 

(Z  .nal)  an  o.n.  set. 
n 

Much  of  the  following  analysis  will  hinge  on  the  properties  of  the  strictly 
negative  eigenvalues  of  S.  These  eigenvalues  will  be  designated  as  Un,  nil), 


! 
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X  s  X  , ,  with  {e  ,  nil}  associated  o.n.  eigenvectors.  Of  course,  depending  on 
n  n+1  n 

the  particular  and  Rw>  the  set  {X^,  nil}  can  be  empty,  finite,  or  countably 
infinite. 

The  joint  measure  on  ^  representing  message  and  channel  output  is 

Pxy,  with  py  the  measure  representing  the  channel  output  process  Y.  They  are 
defined  as  in  [1]: 


Py(B)  =  Px  ®  uN{(x,yj:  A(x)  +  y  e  B} 
Uxy(C)  =  ux  ®  uN{(x,y):  (x,A(x)+y)  e  C} 


where  ux  ®  is  product  measure  on  x  H 2  (all  measures  are  defined  on  the  usual 
Borel  o-fields,  as  in  [1]).  With  these  definitions,  I[X,  A(X)  +  N]  =  I  [PXy] » 


where  I  [uxy]  =  °°  if  uxy 


is  not  absolutely  continuous  with  respect  to  ux  ®  Uy , 


and  otherwise 


l[u  ]  =  / 

H1XH2 


[_dux®uy 


duXY(x,y). 


Two  results  proved  in  [  1  ]  will  be  central  to  the  following  analysis.  The 


first  is  that  for  any  fixed  covariance  operator  R^^  of  the  signal  process,  the 
information  I(uXY)  is  maximized  by  choosing  A(X)  to  be  Gaussian  [1;  Lemma  6]  . 

The  second  is  that  if  A(X)  is  Gaussian  with  covariance  operator 


■W)  ■  ^  v,i  •  [Ri/2  vji 

where  J  a.,  <  °°  and  {vn>  nil}  is  an  o.n.  set  in  ,  then  I  [uXy]  *  (1/2)  J  log[l*an] 

n 

[1,  pp.  83-84]. 

Finally,  it  is  noted  that  |  |R^^x|  ij  =  I  | x f  lw  can  be  viewed  as  the  norm  of 
x  in  the  RKHS  defined  by  the  kernel  rw(t,s):  -♦1R,  rw(t,s)  ■  <Rwt,s>2. 

rw  actually  determines  a  RKHS  of  real-valued  functions  on  Hjl  however,  this  RKHS  is 
a  subset  of  the  bounded  linear  functionals  on  H^,  and  thus  can  be  regarded  as  a 


subset  of  H2. 
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Main  Result 

The  solution  for  the  information  capacity  problem  defined  in  the  preceding 
section  is  given  in  the  following  theorem. 


Theorem .  Let  C  =  sup  I  [vYV] ,  where  £  is  defined  by  assumptions  (1)  and  (2) 

d 

of  the  preceding  section. 


(a)  If  is  finite-dimensional,  then 


C  =  %  l  log 
n=l 


r  K 

I  (l+Y-)+f 
i  =  l  1 


K(!+Yn) 


where  K  is  the  largest  integer  such  that  £  y  +  P  >  Ky,,.  and 

j  T1  K 


Yj  s  y2  <  ...  is  the  set  of  all  eigenvalues  of  S. 


(b)  If  H  is  infinite-dimensional,  {X  ,  nil}  is  not  empty,  and  £  X  P  >  KX 


for  all  negative  eigenvalues  X  of  S,  then  J  |X  |  <,  P  and 


C  =  % 


Vog  fcU  * 1/2 


(c)  If  H2  is  infinite-dimensional,  {X^,  nil}  is  not  empty,  and  there  exists  a 


largest  integer  K  such  that  J  X.  ♦  P  >  KX  and  X„  <  sup{X  ,  nil}  then 

•  i  i\  iv  n 


C  =  hi 


K  ,  i*l 

n-1  *  L  KCl-Xn) 


(d)  If  Hj  is  infinite-dimensional,  and  (Xn>  nil}  is  empty,  then  C  *  P/2. 


In  (a)  and  (c),  the  capacity  can  be  attained.  In  (a),  it  is  attained  using 

a  Gaussian  signal  with  covariance 

ni?l/2  ..  ,  .  rol/2 
n»l 


V)  -  vii  •  ni' 
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-,P.v 


where  {vj,...,vK}  is  an  o.n.  set,  Svi  =  y^,  and  tr=  (1+Yn)  J- — £ - YnJ  • 

The  capacity  in  (c)  is  attained  with  a  Gaussian  signal  having  covariance  of  the 
same  form  as  that  for  (a),  but  with  y^  replaced  by  and  v^  replaced  by  e^, 
i — 1 ,  •  • » ,  K  • 

In  (b)  and  (d) ,  the  capacity  cannot  be  attained,  except  in  (b)  for  the  special 


case  where  P  ■ 


-2»v 


In  (b)  and  (c),  the  capacity  is  strictly  greater  than  the  capacity  obtained 
using  the  constraint  Ey  |  |A(X)  |  |^S /H  |  |r^^2A(x)  |  |2  dpx(x)<P;  in  (d) ,  the  capacity 

is  the  same.  In  (a),  the  capacity  is  strictly  less  than  that  for  the  constraint 

2 

E  | |A(X) lisp  if  all  y  20  with  >  0;  strictly  greater  if  all  y  <  0  and 
N  n  k  n 

Yj  <  0;  and  no  general  statement  holds  if  y^  >  0  and  y^  <  0. 


Proof  of  the  Main  Result 


The  proof  of  the  Theorem  will  be  given  after  having  obtained  several  lemmas. 


Lemma 


1  Let  N  2  1  be  fixed,  and  suppose  {pn,  n*l,...,N}  is  a  set  of  strictly  pos¬ 


itive  scalars,  2  p^  2... 2  p^.  Let  A  c  R  be  the  set  of  all  x  such  that  x  2  0  for 

k  k  N  N 

n*l...,  N,  and  II  x  s  II  p  for  k=l...,  N.  Then  sup  JI[l+x  ]  *  JI[l+p  ]. 

1  n  1  n  All 

N  N 

Proof  Define  GN:  It  ■*  X  by  GN (X)  *  l  log[l+xnJ.  It  is  sufficient  to  show  that 

GNQ()  s  GN(£),  subject  to  the  constraints 


-xn  *  0 


n*l...,  N 


J  log  xn  -  1  log  pn  s  0  k»l, . . . ,  N  (C2). 

N  N 

The  constraints  C^nC^  define  a  convex  set  in  It  ,  and  G^  is  concave  on  IR  . 

Thus,  one  has  the  problem  of  maximizing  a  concave  function  over  a  convex  set  in  1RN, 
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with  concave  and  differentiable  constraints;  by  Kuhn-Tucker  theory  [7]  any 

solution  to  this  problem  will  define  a  global  maximum  over  the  set  A.  A  solu- 

.  N  .  M 

tion  is  any  x*  inlR  satisfying  the  following  set  of  equations  for  some  jg  in® 

such  that  y.  $  0  and  6  <  0  for  i=l,..,N  [7): 


1+x 


1  1  r  -  . 

x*  “  Yn  +  x*  ^  ®k  *  °’ 


n  k^n 


n=l , . . ,N 


(sl) 


VS  *  0  and  -x*  s  0, 


n=l, . .  ,N 


(s2) 


k  k 

log  x*  -  £  log  p  i  ]  =  0 
^  2  1  ** 


l  log  x*  -  l  log  p  <  0 
1  n  1  n 


k*l, . .N 


(s3). 


The  system  of  equations  (Sj),  (S2) ,  (S3)  is  easily  seen  to  be  solved  by  taking 

N 

x*  =  Pn>  Yn  =  0,  and  J  6^  =  -pn/(l+pn)  for  n=l,..,N.  The  fact  that  this  solu¬ 
tion  gives  6^  s  0  for  all  k=l,..,N  follows  by  induction  on  N  -  k+1,  using 
Pn  i  Pn_j  for  2  <  n  s  N.  □ 


2  2  2 
Lemma  2  Let  (xn>  nail}  be  a  summable  sequence,  with  xn  a  x^+j.  Suppose  that 

S  is  a  compact  operator  in  H2  which  is  symmetric,  has  M  £  00  strictly  negative 

eigenvalues,  and  I  +  S  is  strictly  positive.  Then  for  any  finite  N  s  M  and  any 

o.n.  set  {u  j, . .  .u^}, 

n[i«.x*||(i+s)1/2un|r2]  S  n[i+x2(i+xn)_1i 

where  X  *  X  *  . .  are  t*ie  negative  eigenvalues  of  S. 


Proof 


Using  the  inequality  1 1  (I+S)  1/2un 1 1"2  s  1 1  (I*S)'1/2un 1 12.  it  suffices 


to  show  that  n[l*xj||(l*sr1/2unl|2]  * 
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Let  {un>  nil}  be  any  fixed  CON  set  in  H r  Define  X  =  £|x  ju  ®  u  .  The  previous 

n 

inequality  will  be  proved  if  one  shows  that 


N  1/7  ,  N  _ 

n[i+l I (i+s)"i/2x  un||‘]  <  n[i+xJ(i+An)-1] 


By  a  result  of  Horn  [5], 


J  I !  a+s)"1/2x  un| |2  s  jj  Tn(p) |xn|2'1/4P 


for  all  k  i  N  and  any  fixed  pal,  where  {t  (p),  psl }  are  the  eigenvalues  of 

X  ^  ^(I+S)  X  '  P  and  tn(p)  i  Tn+l^‘  follows  from  the  fact  that 

(I+S)  1/2x1//2P  is  compact,  as  is  x1_1^2p  =  £|x  |1_1/2P  u  ®u  . 

Let  a  >  0  be  given.  For  sufficiently  large  p,  (1+a) (I+S)  >  X*^2P(I+S) 

k  k  k  1 

and  then  n  t  (p)  <  (1+a)  n  (1+A  )  ,  so  that 

1  1  n 

k  k 

J  I  l(I+S)"1/2X  uR|  ]2  <  (l+a)k  II(UAn)-1|xn|2‘1/4P,  k  =  1,2, ...  ,N.  Thus,  for 
all  a  >  0,  n  ||(I+S)'1/2Xu J|2  <  (l+a)k  n(l+An)_1x2,  giving 

2  **  ■  n  ji 

k  y  k 

n  | | (I+S)  1  2  x  un i 1 2  s  n (1+An)  1x2,  for  k=  1,2,..,N.  Applying  Lemma  1,  this 

N  ,,,  ?  N  ,  . 

yields  n[l+||(I+S)“4/^Xu  1 1‘]  s  nfl+x-d+Aj"1].  □ 

1  ^  |  n  n 


Lemma  3  Suppose  that  all  the  eigenvalues  {An,  nil}  of  S  are  strictly  negative, 
Aj  s  A^  s  - -  with  Se^  *  A^e^,  i  1  l  and  {e^,  iil}  an  o.n.  set. 


(a)  If  there  exists  a  largest  integer  Kil  such  that  [  A  +  P  >  KA„,  then 

r  a 


c  *  h  l  log 

n*l 


l  d^i)  ♦  P 
i«l 

.  ‘o-V  . 


The  capacity  can  be  achieved;  it  is  attained  with  a  Gaussian  signal  having  co- 
variance  operator 

,1/2.  ,  .  rol/2. 


"A(X) 


-  I  •  iC\] 
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(1+A.)"1  K 

where  t.  =  - —  (£  A  +P  -  KA.),  i=l,..,K. 

1  *  n=l  n  1 

This  result  includes  the  case  when  is  finite  dimensional  (so  that  K  s  dimf!^)). 

Moreover,  this  result  is  also  obtained  when  one  adds  the  additional  restriction 

-1  M 
that  support  (yx  °  A~  )  <  M  <  °°.  With  this  restriction,  K  =  M  if  £  Ar  +  P  >  MAM; 

otherwise  there  exists  K  <  M  such  that  the  above  expression  is  the  capacity. 


(b)  If  there  does  not  exist  a  largest  integer  K  such  that  £  A  +  P  >  KA  , 

^  n  i\ 


then  I  l\,  I  s  P,  and 


oo  p 

c  =  *  I  log  I+T  +  h  (P  +  l  V* 

n=l  L  nJ  n 

Capacity  can  be  attained  only  if  P  =  An;  it  is  then  attained  by  a  Gaussian  sig¬ 
nal  with  covariance  RAX  =  \  t.JRj'VV]  ®  [R^U*ei]  where  Ti  =  -j^XT  •  If  P  >  A^, 


then  the  capacity  is  the  limit  of  the  mutual  information  for  a  sequence  of  Gaussian 

w 

_M 

..  j  t.i 7  T  n  /tmra  vi  oti  «/»  . 

maxj 


signals  (y^),  with  covariance  R^x  =  \  [R^U*^]  ®  [R^U*e^]  and 


M 

T.  = 
1 


U  +  M 


•A.)'1  M 

w - (I  An  +  P  -  MXi)  i=l, . .  ,M. 


Proof 

OO 

From  the  results  of  [1,  pp.  83-84],  I[yXYJ  *  J  log[l+Tn]  when  yx  «  A" 

n«l 

is  Gaussian  with  covariance  operator 

Vi*  Cr^/2u»l  *  ir"/2u"' 

with  (u^,  nil)  any  o.n.  set  in  By  Assumption  (1),  R^2  ■  R*^2  (I+S)^2U* 
with  U  unitary.  Thus 

El I^J1/2A(X) | |2  >  Trace  Rjj1^2RA(X)Rj^2 

■  Trace  (I+S)*U*R^*  RA(X)RjfFlJ 
•I  tJ  |CI*SJ*U*wBl||. 
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Moreover,  for  any  choice  of  covariance  operator  RA^,  the  information  is  maximized 

if  •  A  ^  is  Gaussian  [1,  Lemma  6].  Thus,  one  can  assume  that  yx  °  A*  is 

Gaussian.  The  capacity  problem  now  reduces  to  finding  C  =  sup  %  £  log[l+x  ], 

S'  n  n 

where  O'  is  the  set  of  all  {(xj,  (U  J  }  such  that  x  £  0  for  n  >  1,  T  t  <  <*>, 

nn  n  Ln 

n 

{u  ,  n^l}  is  an  o.n.  set,  and  7  x  I  I  (I+S)^U*u  I  I ^  <  P . 

n 

We  rewrite  the  preceding  expression  as 


C  =  sup  l  log [1  ♦  x2  ||(l4S)%U*un||:2], 
Q'  n  n  ^  2 


2  7  2  2 

where  xn  =  xn  |  |  (I+S)?U*Un |  ^  Since  {xn,  nil)  and  {x^,  n2l)  are  summable,  and 

(u_>  n>l}  is  to  be  selected,  one  can  assume  WLOG  that  x'a  x  , ,  nil.  From  Lemma  2, 
n  n  n+1 


for  any  such  choice  of  {x^,  l<n<M}, 


^  log[l  +  xn||(I+S)£U*un||-2]  s  l  logfl  +  x^d^)*1]. 


We  will  maximize  the  right  side  of  this  inequality  for  fixed  M,  and  show  that  the 

2  M  2  2  2 

maximum  can  be  attained  by  {x  ,  l<n<M}  such  that  7  x  =  P  and  x  >  \  .  for 

n  ‘'i  n  n  n+1 

n=l , . . ,M. 

M  ** 

For  fixed  M  >  1,  define  f^:  1R  -+1R  by  fM(y)  =  I  log[l  +  yn(l+An)  ].  We 

n=l 

seek  to  maximize  fM(y)  subject  to  the  constraints 

M 

8(x)  E  I  yn  ■  p  -  °» 

1  n 

hi(X)  E  -Yi  *  0.  i=l , . . ,M  , 

This  is  a  constrained  maximization  problem.  Since  log(l  +  ay)  is  concave  over 
{y:  yiO)  for  any  a  >  0,  the  function  fM  is  concave  over  the  convex  set 
(Z  inlR  :  *  0,  i«l,..,M}.  Moreover,  each  constraint  function  is  concave. 

Thus,  any  solution  to  this  problem  will  define  a  global  maximum  for  f^  [7].  In 
order  for  y*  to  be  a  solution,  it  is  necessary  and  sufficient  that  the  following 
set  of  equations  be  satisfied  [7]  : 


>  ! 


V 


».«»***»  tt  *V- 
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then  constraints  (l)-(3)  are  satisfied  by  choosing 

K  , 

8  =  -K[P  ♦  K  ♦  l  A.  ]  1 
1  1 


yi  =  o,  i  >  k 

i  K 

=  K  [P  +  l  a  -  KA  J  ,  isx 

1  n=l 

K  1  1 

Y  =  -K[P  ♦  K  +  l  A  ]'1  *  (1+A  )_1  i>K. 


Thus,  under  the  assumptions  of  (a). 


sup  l  log[l  +  x^(l+A  )_1]  =  l  log'  1-1 
Q"  1  n  n  n=l 


j  A. +  P+K 

L-y  i 


K(l+An) 


where  Q"  =  Uxn):  I  xn  s  P).  As  alrea<^y  noted,  when  yx  ° 


covariance  operator  R 


I,  <V 


,1/2, 


A(X) 


A  1  is  Gaussian  with 
n>l)  an  o.n.  set,  then 


w  w 

I[uXYl  =  %  l  log[l+Tn]  S  h  l  log[l  +  x^(l+An)_1] 


n=l 


n=l 


if  *1  =  I  I  (I+s)%U*un )  xn  2  Vr 


n  n 
Choosing  u  =  U  e  ,n2l. 


<n  =  K_1[P  ♦  l  A  -  KA],  n-l,..,l 
n  i=l  1  n 


2 

xn  B  0,  n>k 

one  obtains  (a)  immediately  if  is  finite-dimensional.  If  is  infinite-dimen 

sional,  then  (a)  is  obtained  as  above  by  taking  M  sufficiently  large,  and  noting 
M 

that  l  log[l  ♦  t  ]  is  a  non-decreasing  function  of  M  if  t  i  0  for  nil.  The 
j  n  n 

proof  of  (a)  under  the  additional  constraint  that  support  (y^  «  A"1)  s  M  <  ®  can 
be  obtained  from  the  preceding  proof  and  the  results  of  [1,  pp.  83-84]. 
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Suppose  now  that  £  X  +  P  >  KA  for  all  K  £  1.  If  P  <  J  |A  |, 
i  n  k  .  n 

K 

there  exists  K  2  1  and  A  >  0  such  that  P  +  l  Ar  =  -A.  Thus, 

K 

I*k+1I>U/K)[-Z  Xn  ‘  P]  =  A/K- 

Assume  |A  |  >(1/K)A  for  1  <  p  <  N. 

K+p 


K+N+l 


1  r  k+n  1  if  k+n 

>  -ri-rr  -F  A  -  P  =  A  -  F  A 
K+N  L1  n  K+N  iK+1  n 

>  J  u  _ 


so  that 


IWlI  "  K^N  [A  +(N/K3A^  =  A/K- 


Hence,  | A„  |  >  A/K  for  all  p  2  1,  which  contradicts  Afi  -*•  0,  proving 


\  ix„i  s  p- 


A  lower  bound  on  C  under  the  assumptions  of  (b)  is  now  obtained  from  the 
proof  of  (a),  by  taking  CK  to  be  the  value  of  C  under  the  additional  constraint 
that  support  (yx  •  A*  )  has  dimension  i  K,  so  that 


r  K 

K  I  A.  ♦  P 


and  so 


C  a  lim  C  =  hi  log[(l*A  )  j  ♦  h(P  +  l 

K  *  1  n  1  n 

To  see  that  lim  C„  is  an  upper  bound  for  C,  suppose  that  lim  C„  <  C.  Then  there 

K  -1  K  K 

exists  a  Gaussian  «  A  with  covariance  operator 


OO 

Vx)  ’  \  with 

OD 

l  Tn 1 1 (I+S)^U*u^ 1 1 ^  s  P.  and  for  some  finite  M 


>'  « 
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h  l  log [1  +  t']  >  lim  C 


However,  as  already  seen,  no  selection  of  (u^) }  satisfying  the  above  condi¬ 

tions  can  be  such  that 
M 

h  l ^  log [1  ♦  T^]  >  cM. 

This  contradiction  establishes  (b) .  The  fact  that  the  capacity  in  (b)  cannot 

be  attained  follows  in  the  same  way.  □ 

Lemma  4  Let  C,.,,  =  sup  ,  t  I[pvv]- 

-  (N)  {(A.ux):  E  | |R^1/2A(X) I  I*  *  P)  XY 


(a)  If  S  has  only  non-negative  eigenvalues,  then  C  £  C(n)‘ 

(b)  If  S  has  only  non-positive  eigenvalues,  then  C  2  C^. 

(c)  If  H2  is  infinite-dimensional  and  all  eigenvalues  of  S  are  non-negative, 

then  C  =  C,.n  =  P/2. 

(NJ 

Proof  (a)  Here  |  | (I+S) _1 1 j  <1,  and  so  | |R^1/2A(X) | |* 

s  ||(I+S)'l||  I |R^1/2A(X) I l2  -  ||R‘1/2A(X)||2 


for  all  A(X)  in  range  (R*^).  Hence  |  |R^1/^A(X)  |  |2  s  P  implies 

Ep  I  |RnV2A(X)  I  s  P,  so  that  C  is  obtained  by  a  supremum  over  a  smaller  set, 
yielding  (a) . 


-1/2, 


(b)  In  this  case  ||l+S||  s  1,  and  so 

,-1/2. 1  |2 


Ey  I |R^i/^A(X) | I2  *  P  implies  Ey  | |R^1/2ACX) | s  P. 

X  X 

(c)  From  (a)  and  [1,  Theorem  2],  C  i  P/2.  For  fixed  nil,  let  yA  .  be 

n1 


C_  n 


Gaussian  with  covariance  (X  )  *  ~n  ^  ^RN^2ui^  *  ^^i^  with  ^ui* 

n  1 


,l/2„ 
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a  CON  set  in  H..  From 


2<  From  [i.  Theorem  1],  I y  ]  =(n/2)log[l  «•  — ] .  Define  (Cn)  by 

n  n 


Cn  -  nP[f  ||(I+S)*U*u  llj]'1-  Then  Ey  I  |R^1/2A(Xn)  |  j  =  P,  so  (A,px  )  c  Q.. 

1  X  21 

n 

n  %  2 

The  fact  that  (l/n)J  |  |  (I+S)  u*u  I  U  "*■  1  follows  easily  from  the  fact  that 
i=l  1  1 

|  |s1/2u.  |  \2  ■+  o.  Thus,  C  -^P,  and  so  I[nY  v  ]  +  P/2,  showing  C  ;>  P/2. 

1  1  n  Vn 


□ 


Proof  of  Theorem 


(a)  is  proved  exactly  as  (a)  of  Lemma  3.  (d)  is  contained  in  Lemma  4. 

To  prove  (b)  and  (c)  we  identify  two  possibilities  for  each:  (1)  S  has  a  finite  set 
of  M  strictly  negative  eigenvalues;  (2)  S  has  an  infinite  set  of  strictly  neg¬ 
ative  eigenvalues.  We  prove  only  (b)  and  (c)  for  case  (1);  the  proof  for  (2)  is 
similar  but  simpler. 

First,  we  note  that  the  capacity  is  at  least  C,  as  given  in  (b)  and  (c). 

This  follows  immediately  in  (c)  using  a  Gaussian  pax  with  covariance  as  specified 
for  (c)  in  the  theorem.  For  (b),  this  is  shown  by  using  a  sequence  of  Gaussian 
signals  (uAX)  having  covariance 


AX 


M 

l r £  <\  •  *  -rir^  lml  4%  •  -*• 


S  e.  *  A.e.,  \.<  0,  S  u.  =  y.u.,  y.  z  0,  for  i«l,..,M;  i*M+l,...,k;  and  {e.,..,eu, 
1  1  i  1  J  J  J  k  M  i  M  1  M 

uM+l’,''Uk}  is  811  0N  set*  As  k  *»  1  l  lo*[TVT]  *  %CP  ♦  I  aa)  ■  c. 


Thus,  to  prove  (b)  and  (c)  when  S  has  only  a  finite  set  of  strictly  negative  eigen¬ 
values,  it  suffices  to  show  that  the  capacity  is  s  C  as  given  in  the  theorem. 

Suppose  then  that  the  assumptions  of  either  (b)  or  (c)  are  satisfied,  and  that 
S  has  strictly  negative  eigenvalues  s  s...s  ^  Suppose  that  the  true  capacity 
is  greater  than  C,  as  given  by  (b)  or  (c).  Since  the  capacity  is  the  limit  of 
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k  -1 

(I[uxy])  for  a  sequence  (p  ^  °  )  of  Gaussian  measures,  there  exists  [1] 

*1  1/2  1/2 

a  covariance  operator  R  such  that  R  =  £  R^  u^  e  R*'  u ^ J  log[l  +Tj]  >  C, 

«J  r.  2 

{u  ,  jai}  an  o.n.  set,  and  \  x.  |  |  (I+Syll*u.  |  |  =  P.  s  P,  We  can  assume  that 

1  j  3  3  +  1 

J  >  M.  Let  T:  •+  H2  be  the  unitary  map  defined  by  TVj=ll*Uj,  where  {v^ ,  j£l) 

2  %  2 

are  the  eigenvectors  of  S.  Defining  xf  =  Tj  |  |  (I+SyU*Uj | |2>  one  has  that 

l  log[l  +  x.]  *  l  log[l  +  x? j | (I+S) *^U*u . 1 1 ?] 

1  1  J  J 

=  l  log[l  +  x2||(I+S)^U*Tv.||2]. 

We  can  assume  that  xf  i  x2  ,  )=!>••*  >1*1;  from  Lemma  2, 

J  J  * 

J  M  2  -1 
J  log [ 1  +  x.]  <  l  log [1  +  x  (1+X.)  ] 

1  1  J  J 


*1  log[l  *  x2||CI+S)-1/2Tv.||2] 

Mj.  1  J  J 


M+l 


M 


Define  P  ' =  £  xf.  Then  J  xf  *  P.  -  P  the  eigenvalues  of  (I+S)  are  the 
1  1  J  M+l  3  A  1 


same  as  those  of  T*(I+S)  ^T,  and  thus  Lemma  3  and  Lemma  4(a)  yield 


J  K 

l%  log[l  ♦  T.]  s  ^(Pj-)  =  l  log 


7KU*Xi)  ♦  P2' 
1 


K(l+X  ) 


♦  P  -  P  ' 
*1  *1  * 


We  maximize  C^Pj')  with  respect  to  P^.  The  derivative  is  non-decreasing  for 
K 

increasing  Pj'  if  £  A.  +  P^  s  0.  This  is  satisfied  if  K  <  M;  taking  Pj  arbitrarily 
close  to  P  (J  ■*  “),  one  obtains  part  (c)  of  the  theorem.  If  K  «  M,  then 

arrtyPj')  >  0  if  Pl'  <  V  <  0  if  pi'  >  -F  Md  by  continuity 

M  1 

of  CQ,  CqIPj')  i  CQ(-l  Aj),  proving  (b)  of  the  theorem. 

Hie  remaining  parts  of  the  theorem  can  be  obtained  from  the  proof  of  Lemma  3, 
from  Lama  4.  D 
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